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Abstract 

We analyse the prospects to detect at the LHC the neutral Higgs particles of the 
Minimal Supersymmetric Standard Model, when the masses of the two CP-even h, H 
and of the CP-odd A boson are close to one another, and the value of tan (3 is large. 
In this "intense-coupling regime" , the Higgs bosons have strongly enhanced couplings 
to isospin down-type fermions and large total decay widths, so that the 77, WW* and 
ZZ* decay modes of the three Higgs bosons are strongly suppressed. We advocate the 
use of the decays into muon pairs, h,H,A^ A*"*"/^") to resolve the three Higgs boson 
peaks: although the branching ratios are small, C(IO^), the resolution on muons is 
good enough to allow for their detection, if the mass splitting is large enough. Using 
an event generator analysis and a fast detector simulation, we show that only the 
process pp — > bb^~^fi~ , when at least one of the 6-quarks is detected, is viable. 
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1. Introduction 



The search for the Higgs bosons and the study of their fundamental properties are the 
primary goals of the LHC. To make sure that Higgs particles with masses in the vicinity 

of the electroweak scale cannot escape detection, two benchmark models have been studied 
in great detail: the Standard Model (SM), which predicts the existence of a single Higgs 
particle ^ and its minimal supersymmetric extension (MSSM), where the Higgs sector is 
extended to contain two CP-even Higgs particles h and if, a CP-odd or pseudoscalar Higgs 
boson A, and two charged Higgs particles [1]. In the case of the SM Higgs particle, a 
plethora of production channels can be used at the LHC and one of the main detection modes 
is expected to be the gluon-gluon fusion process, gg with the signatures — > 77 or 

ZZ^*\ WW — > M in, respectively, the low and high Higgs boson mass ranges [2]. 

Because of the complexity of the Higgs spectrum, the situation is more complicated in 
the MSSM and depends on the values of the two input parameters that characterize the 
Higgs sector at the tree level, the pseudoscalar Higgs mass Ma and the ratio of the vacuum 
expectation values of the two Higgs doublet fields tan/3. It depends also on the mixing 
in the scalar top sector, which is controlled by the trilinear coupling Af, when radiative 
corrections are included [3] . The latter push the maximal value of the lightest h boson from 
M;^^^ ~ \cos2/3\Mz < Mz at the tree level, to M^°^^ ~ 110-130 GeV, depending on the 
values^ of tan f3 and At. The MSSM Higgs sector can be divided into three regimes, according 
to the relative magnitudes of the pseudoscalar Higgs mass and the maximal value of the 
lightest h boson mass MJ^^. For large tan/3 values, for which M^™ ~ M^^, the search at 
the LHC can be summarized as follows [for details, see Ref. [2] for instance]: 

(i) Ma ^ M™^: in this case, we are in the so-called decoupling regime, in which the 
H,A and bosons are very heavy and almost degenerate in mass. Ma ~ Mh ~ Mh±, 
while the h boson has a mass M^ — M™'^^ and SM-like Higgs properties. The techniques 
devised for the detection of a light particle can be adapted to the h boson. The A and 
H bosons, if not too heavy, can be searched for in the channels gg/qq ^ bbH/A — > bbr^r^ , 
while the if^ particle can be detected in the process gg/qq ^ tbH" — > tbTUr for instance. 

(ii) Ma < MJ^^: in this case, it is the heavier H boson that will be SM-hke, while the 
h and A particles will be degenerate in mass and couple strongly to 6-quarks and r-leptons 

for large values of tan /3. The search techniques are the same as above, except that the roles 
of the h and H bosons are interchanged. The if + particles, because of the MSSM relation 
Mh± ~ \J M\ + M^, are light enough to be detected in top-quark decays, t H^b. 

{Hi) Ma ~ M™^^: this is what was called the "intense-coupling regime" [6] where the 
three neutral Higgs bosons have comparable masses, M/j ~ Mh — Ma- The couplings of the 
CP-even Higgs particles to gauge bosons are both suppressed with respect to the SM [the 
A boson does not couple to gauge bosons because of CP-invariance] , while the h, H and A 
couplings to doAA-n-ty])e (u])-ty])c) fermions are strongly enhanced (suppressed). 

""^ Values tan (3 > 3-10, depending on the mixing in the scalar top sector, are required to maximize the h 
boson mass and to evade the experimental constraint from LEP2 searches, M^, ~ Ma ^ 92 GeV [4]. From 
the theoretical viewpoint, very large values of tan/3 ~ nit /mi, ~ 0(50) are very interesting, since they allow 
for Yukawa coupling unification at the Grand Unification scale; see Ref. [5] . 
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While detailed experimental analyses have been performed for the first two scenarios [2] , 
only little work has been done for the intense-coupling regime. In the detailed theoretical 
discussion given in Ref. [6], it was shown that the search at the LHC might be rather 
difficult in this regime. The main problem is due to the fact that, for ~ Mh ~ Ma and 
tan /5 3> 1, the three neutral Higgs bosons will mainly decay into isospin down- type fermions 
and the clear 77 and ZZ*, WW signatures cannot be used anymore, the branching fractions 
being too small. In addition, since the Higgs masses are close, it will be difficult to detect 
individually the three Higgs bosons, and resolving between the peaks is made even more 
difficult since the total decay widths can be rather large, implying broader signals. 

In this note, we discuss the detection of the three neutral MSSM Higgs bosons ^ = h, H, A 
in this scenario, paying a special attention to the possibility of resolving the signal peaks. 
Performing an event generator analysis that takes into account the signals and the various 
backgrounds, as well as a simulation of some aspects of one of the LHC detectors [CMS] 
response, we show that the detection of separate Higgs bosons can be extremely difficult. 
It can be done only if the Higgs mass differences are sizeable and only if the rare decays 
into muon pairs, $ /^'''/i", which have branching ratios at the level of a few times 10~^, 
are exploited. In addition, the Higgs bosons need to be produced in the associate processes 
gg — > 66$, i.e. with at least one 6-quark being detected; the processes gg ^ ^ and 66 — > $ 
suffer from the very large background from Drell-Yan production pp — > 7*, Z^*'^ — > jJi,'^ jJ," . 

The rest of the discussion is as follows. In the next section, we will recall the main features 
of the intense coupling scenario. In section 3, we discuss the production of the neutral Higgs 
bosons and their main backgrounds at the LHC. In section 4, an event generator analysis 
for the separation of the Higgs bosons is presented. A short conclusion is then given. 

2. The intense-coupling regime 

As introduced above, the intense-coupling regime is characterized by a rather large value of 
tan /9, and a pscudoscalar Higgs boson mass that is close to the maximal (minimal) value of 
the CP-even h [H] boson mass. Ma ~ almost leading to a mass degeneracy of the 

neutral Higgs particles, Mh ~ Ma ~ Mh- In the following, we will summarize the main 
features of this scenario. For the numerical illustration, we will fix the parameter tan (5 to 
the value tan/5 = 30 and choose the maximal mixing scenario, where the trilinear Higgs- 
stop coupling is given by At — ^/QMs with the common stop masses fixed to Ms = 1 TeV; 
the other SUSY parameter will play only a minor role. The determination of the Higgs 
masses, couplings and branching ratios is performed using the program HDECAY [7] in which 
the routine FeynHiggsFast [8] is used for the implementation of the radiative corrections. 

The left-hand side of Fig. 1 displays the masses of the MSSM Higgs bosons as a function 
of Ma, the latter varying from 100 to 140 GeV for our representative value of tan/3 in the 
scenario of maximal stop mixing. As can be seen, for Ma close to the maximal h boson 
mass, which in this case is M™^^ ~ 130 GeV, the mass differences M4 — and M^ — Ma 
are less than about 5 GeV. The H boson mass, given by Mfj± ~ M| + M^, is lar ger : m 
the range Ma ^ 140 GeV, one has Mh± ^ 160 GeV, implying that charged Higgs bosons 
can always be produced in top-quark decays, t — > H~^b, and be detected at the LHC. 
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The couplings of the CP-even Higgs bosons to fermions and gauge bosons normahzed 
to the SM Higgs boson couphngs are shown in the right-hand side of Fig. 1 for the same 
inputs as previously. For small values, the H boson has almost SM couplings, while the 
couplings of the h boson to W,Z,t (6) are suppressed (enhanced); for large values the 
roles of h and H are interchanged. For medium values. Ma ~ M™^^, the couplings of both 
h and H to gauge bosons V = W, Z and top quarks are suppressed, while the couplings to b 
quarks [for which 10 x g^^f^ are shown in the figure] are strongly enhanced. The normalized 
couplings of the CP-even Higgs particle are simply Qavv = and gAbb = ^1 QAtt = tan /5 = 30. 
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Figure 1: The masses of the MSSM Higgs bosons (left) and the normalized couplings of the 
CP-even Higgs bosons to vector bosons and third-generation quarks (right) as a function of 
Ma and tanjS = 30. For the b- quark couplings, the values 10 x (7^^^ are plotted. 

These couplings determine to a large extent the branching ratios of the Higgs particle 
decays, which are shown in Fig. 2. Because the couplings of the three Higgs particles to 
6-quarks and r-leptons are strongly enhanced, their branching ratios to bb and t^t~ final 
are the dominant ones, with values ~ 90% and ~ 10% respectively. The decays H —>■ WW* 
do not exceed the level of 10%, even for small M4 values [where H is almost SM-like] and 
in most of the range displayed for M4, both the decays H,h ^ WW* are suppressed to the 
level where they are not useful. The decays into ZZ* are one order of magnitude smaller. 
The interesting rare decay mode into 77, which is at the level of a few times 10~^ in the 
SM, is very strongly suppressed for the three Higgs particles and cannot be used anymore. 
Finally, note that the branching ratios for the decays into muons, $ f^'^fJ'~, are constant 
in the entire Ma range exhibited, at the level of 3 x 10"*^. 

Summing up the partial widths for all decays, the total decay widths of the three Higgs 
particles are shown in the left-hand side of Fig. 3. As can be seen, for Ma ~ 130 GeV, they 
are at the level of 1-2 GeV, i.e. two orders of magnitude larger than the width of the SM 
Higgs boson for this value of tan/5 [the total width increases as tan^/5]. The right-hand side 
of the figure shows the mass bands M$ ± r$ and, as can be seen, for the above value of Ma, 
the three Higgs boson masses are overlapping. 
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Figure 2: The branching ratios of the neutral MSSM Higgs bosons h, A, H for the various 
decay modes as a function of Ma and for tan(3 = 30. 



- r$ [GeV] 

tan /9 = 30 



0.1 r 



0.01 



100 




10 X r^o 



Ma [GeV] 
I 



110 



120 



130 



140 



- 130 



. 120 



110 



100 



1 1 

- M$ ± r$ 


TT 1 

''''''"'a/ / 


tan /? = 30 








/ 1 1 


Ma [GeV] 
1 



140 100 



110 



120 



130 



140 



Figure 3: Total decay widths r$ (left) and the mass bands M$ ± r$ (right) for the neutral 
MSSM Higgs bosons as a function of Ma and for tanf3 = 30. 

All these points are summarized for the three values Ma = 125, 130 and 135 GeV in 
Table 1, where we display the Higgs boson masses, their total decay widths and the branching 
ratios of some important decay modes. These three points, called respectively PI, P2 and 
P3, are the ones we will choose to perform our analysis of the production and the detection 
of these Higgs particles at the LHC, the subject to which we now turn our attention. 
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6.5 X 10-^ 


3.35 X 10- 


-4 



Table 1: Masses, total widths (in GeV) and some decay branching ratios for the points PI, 
P2 and P3. The cross sections for the processes pp ^ and $66 (in pb) are also shown. 

3. Signals and backgrounds at the LHC 

As discussed above, the most difficult problem we must face in the intense- coupling regime, 
is to resolve between the three peaks of the neutral Higgs bosons when their masses are close 
to one another. The only decays with large branching ratios on which we can rely are the 66 
and T+r" modes. While the former has too large a QCD background to be useful, the latter 
channel has been shown to be viable for discovery^ [2]. However, the expected experimental 
resolution on the invariant mass of the t^t^ system, in the mass range that we are interested 
in, is of the order of 10 to 20 GeV, and thus clearly too large to resolve the three Higgs peaks. 
One would then simply observe a relatively wide resonance corresponding to A and h and/or 
H production. Since the branching ratios of the decays into 77 and ZZ* — > M are too small, 
a way out is to use the Higgs decays into muon pairs: although the branchings ratio arc 
rather small, BR($ fJ'^fJ'^) ~ 3.3 x lO^^, the resolution is expected to be as good as 1 
GeV, i.e. comparable to the total width, for M$ ~ 130 GeV. 

Because of the strong enhancement of the Higgs couplings to bottom quarks, the three 
Higgs bosons will be produced at the LHC mainly in the gluon-gluon process 

gg^^^h,H,A^ pL+fjL- , (1) 

which is dominantly mediated by 6-quark loops, and the associated production with 66 pairs, 

gg/qq^bb + ^ ^h,H,A^bb + iJi^lJi'' . (2) 

The Higgs-strahlung pp — > HV and vector-boson fusion qq — > Hqq processes, as well as 
associated production with top quarks, will have smaller cross sections than for the SM 
Higgs boson since the couplings to the involved particles are suppressed. The two processes 
eqs. (1) and (2) have recently been discussed in Refs. [9] and [10], respectively, and have 

^Notc that in previous CMS analyses of thepp bb+fi.^fjr signature, the complete 4-fermion background 
was not been fully included. This remark applies also to the pp ^ bb + t^t~ signature. 
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been shown to be viable for the discovery of relatively light Higgs bosons^. 

For the calculation of the signal cross sections for the pp —>■ fi~^fi~ final state, since they 
give rise to the same topology, we sum the cross sections of both the gg ^ ^ ^ 
process and the gg/qq ^ bb + ^ process where the transverse momenta of the 6-quarks are 
too small , < 20 GeV, to be detected. For the former process, we apply a i^-factor of 
1.7 [13] to take into account the NLO QCD corrections, while for the latter one, we apply a 
ii'-factor of 1.5 to the LO cross section evaluated at a scale fi ~ as recently reported*^ 
in Ref. [17]. One then obtains production rates at the level of a fraction of a picobarn for 
pp ^ h, H, A ^ fi~^fi~. For the process gg — > $66, where we require the two bottom quarks 
to be detected, i.e. with pY > 20 GeV, the NLO cross section is approximately the same 
as the LO cross section when it is evaluated at the scale fi = Mh/2 and a running 6-quark 
mass for the bottom Yukawa coupling [17]. The obtained cross sections in this case are one 
order of magnitude smaller than those of the inclusive pp ^ ^ ^ fJ'^fJ'^ case. 

For the backgrounds to fj^^fi^ production, we have included only the Drell-Yan process 
pp ^ '-f* , Z* yU+yU", which is expected to be the largest source [as will be seen later, this 
is sufficient for our conclusion]. For the pp — > ^'^fi~bb final state, however, we have included 
the full 4-fermion background, which is mainly due to the process pp bbZ with Z fi~^fi~. 
Both signals and backgrounds have been generated with the program CompHEP [18]. 




50 75 100 125 150 50 75 100 125 150 

M,-,^- [GeV] AI^^^- [GeV] 



Figure 4: The differential cross section in pb/GeV as a function of the dimuon mass for the 
point PI, for both the signal and signal plus background in the processes pp{-^ $) ii~ 
(left figure) and pp{-^ ^bb) fi^fi^bb (right figure) . 



■^The vector-boson fusion process qq — > H'^ has also been considered for a SM Higgs boson 

[11] and has been shown to work at the LHC only if an unreasonable amount of luminosity is collected. In 
addition, the process qq ^ h, H ^ t^t^ discussed in Ref. [12] cannot be used to resolve the two h and H 
peaks in the intense-coupling regime, because of the poor resolution on the r+T~ invariant mass. 

^Note that the cross section calculated directly for bottom-quark fusion 66 — > $ at NNLO [14] gives 
approximately the same results if the factorization and the renormalization scales are chosen properly, as it 
was shown in Refs. [15, 16]. 
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The differential cross sections are shown for the scenario PI as a function of the invariant 
dimuon mass in the left-hand side of Fig. 4, for the final state pp{^ h, H, A) fi^fi~. As 
can be seen, the signal rate is fairly large and we may hope, in principle, to see two peaks: 
one corresponding to the production of the h/A bosons and one to the production of the H 
boson. However, when put on top of the huge Drell-Yan background, the signal becomes 
completely invisible. This holds true even with some optimization of the cuts; for instance, 
applying a cut on the muon transverse momenta pj_ > 50 GeV, which should strongly 
suppress the Drell-Yan cross section, still leaves too large a background. The same features 
hold for the points P2 and P3, and we conclude, contrary to Ref. [9], that already at the level 
of a "theoretical simulation" , the Higgs boson signal in the inclusive pp — > fi~^fi~ process will 
probably be hopeless to extract for Higgs masses below 140 GeV, unless unreasonably large 
values of the parameter tan j3 are chosen to enhance the signal rate. 

In the right-hand side of Fig. 4, we display, again for scenario PI, the signal cross section 
from pp ^~^^~bb and the complete 4-fermion SM background cross section as a function 
of the dimuon system. The number of signal events is an order of magnitude smaller than 
in the previous case, but one can still see the two peaks. However, the main difference here 
is that the rate for the background is much smaller than in the Drell-Yan case. Once the 
signal events are put on top of the background distribution, one can still see the two peaks 
corresponding to h/A and H production. The same analysis has been repeated for scenarios 
P2 and P3, and the output for the signal and background rates is shown in Fig. 5. As 
can be seen, the situation is similar to that of the previous case, except that here the mass 
difference between the Higgs bosons is large enough for us to hope that the three individual 
peaks could be resolved. 

Note, however, that up to now, we did not include any efficiency for the detection of b 
quarks and muons and did not assume any resolution for the mass of the dimuon system. 
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Figure 5: The differential cross section in pb/GeV as a function of Ma for both the signal 
and signal plus background in the process pp{-^ ^bb) — > ii^ii^bb for P2 (left) and P3 (right). 
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4. An event generator analysis 

In order to perform a more realistic analysis, we have generated unweighted events for the 
full 4-fermion background pp — > ii'^ii' + bb and for the signal, for the three parameter points 
PI, P2 and P3 already introduced; as above, the generator CompHEP has been used. With 
the help of the new CompHEP-PYTHIA interface [19], which has been upgraded to include the 
implementation of the so-called Les Houches Accord 1 [20], the unweighted events have been 
processed through PYTHIA 6 . 2 [21] for fragmentation and hadronization, taking into account 
initial- and final-state radiation. We stress, once more, that the requirement of observing 
both 6-jets, with p^** > 20 GeV, leads to a reduction by a factor of 10 of the signal rate 
compared to the fully inclusive case. In turn, the background reduction factor in this case 
is about 200, resulting in a much larger S/y/S + B than in the inclusive case; see Fig. 4. 

To simulate detector effects, such as acceptance, muon momentum smearing, and 6-jet 
tagging, we take the example of the CMS detector. Using the CMS JET package [22], in 
which muon momentum smearing has been parametrized from a full simulation of the CMS 
tracker layout as described in Ref. [23], a mass resolution of about 1% on the dimuons 
from the Higgs decays is obtained. The events are assumed to be triggered by a double 
muon trigger, with a 7 GeV threshold and within an acceptance of |?7| < 2.1, leading to an 
efficiency of 97% per muon [24]. For 6-jet tagging, we use the b-tagging efficiency obtained 
by the technique described in Ref. [25]. This technique, developed for tagging soft jets of 
20-30 GeV, does not require the 6-jet reconstruction in the calorimeter, but exploits the 
tracker information [only searching for vertex and tracks with significant impact parameter] . 
With a full detector simulation, such a method gives 40% efficiency per gg ^ bb + Higgs 
event for a Higgs boson mass of 150 GeV. 

The results of the simulation for an integrated luminosity of 100 fb~^ are shown in Fig. 6. 
As expected, the signal invariant mass distributions become broader, even with the good 
CMS momentum resolution. This is shown in the plots of Fig. 6, where the number of 
H^H^bb events in bins of 0.25 GeV are shown as a function of the mass of the dimuon 
system. The left-hand side shows the signals with and without the resolution smearing as 
obtained in the Monte-Carlo analysis, while the figures in the right-hand side show also the 
backgrounds, including the detector effects. 

For point PI, the signal cross section for the heavier CP-even H boson is significantly 
smaller than the signals from the lighter CP-even h and pseudoscalar A bosons; the latter 
particles are too too close in mass to be resolved, and only one single peak for h/A is clearly 
visible. To resolve also the peak for the H boson, the integrated luminosity should be 
increased by a factor of 3 to 4. In the case of point P2, it would be possible to see also the 
second peak, corresponding to the H boson signal with a luminosity of 100 fb~^, but again 
the h and A peaks cannot be resolved. In the case of point P3, all three h, A and H bosons 
have comparable signal rates, and the mass differences are large enough for us to hope to be 
able to isolate the three different peaks, although with some difficulty. 

Note that, if the r+r~ final-state decays of the Higgs bosons had been used, with the 
expected resolution on r leptons, we would have seen only one broad resonance and could 
not have resolved even two signal peaks in all three scenarios. 
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Figure 6: /i"*"/^" pair invariant mass distributions for the signal before and after detector 
resolution smearing (left) and for the signal and the background (right) for PI, P2 and P3 
parameter points. 
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5. Conclusions 

We have shown that in the intense-couphng regime, i.e. when the h, H and A MSSM bosons 
have masses too close to the critical point M™^ and when the value of tan /3 is large, the 
detection of the individual Higgs boson peaks is very challenging at the LHC. It is only in 
the associated Higgs production mechanism with bb pairs, with at least one tagged 6-jet, and 
with Higgs particles decaying into the clean muon-pair final states, that there is a chance of 
observing the three signals and resolve between them^. This would be possible only if the 
Higgs mass differences are larger than 3-5 GeV. 

In the present note, we only concentrated on the fully exclusive bb + ii'^ii" signature, 
requiring both &-jets to be observed, and included only the irreducible 4-fermion background, 
which is expected to be the dominant one. In a more complete study, one should eventually 
consider the case where only one single 6-jet is tagged, which should increase the cross section 
signal [15], and take into account also the reducible backgrounds from pp — ^ Z*/'~f* fi'^ij,~ 
with mistagged jets [which is expected to be large in this case] as well as other backgrounds. 
Such a study is beyond the scope of this note and will appear elsewhere [27] . 
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